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Abstract. This paper introduces relevant statistics for the description of routes 
in the internet, seen as a graph at the interface level. Based on the observed prop- 
erties, we propose and evaluate methods for generating artificial routes suitable 
for simulation purposes. The work in this paper is based upon a study of over 
seven million route traces produced by CAIDA's skitter infrastructure. 
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1 Introduction 

Realistic modeling of routes in the internet is a challenge for network simulation. Until 
now, one has had to choose one of the three following approaches to simulate routes: (1) 
use the shortest path model, (2) explicitly model the internet hierarchy, and separately 
simulate inter- and intra-domain routing, or (3) replay routes that have been recorded 
with a tool like It racer out elll8l . All of these methods have serious drawbacks. 

The first method does not reflect reality: routes do not in general have the same 
properties as shortest paths, as already pointed out by Paxson 1 26 27 1, because of rout- 
ing policies 1 30 33 1 mainly at the autonomous system (AS) level. As described in detail 
recently by Spring et al. |30|, and earlier by Tangmunarunkit et al. 1341331 . this often 
induces path inflation. The second method is limited by our ability to explicitly simulate 
the internet hierarchy. Much work 1 32 4 1 has been done in order to model the internet 
graph, and much progress has been made, but today's topology generators are still capa- 
ble of being highly inaccurate in capturing some parameters while they strive to adhere 
to others. (See, for instance, the findings in Li et al.'s Sigcomm 2004 paper 1231 .~) Then, 
even if one is satisfied with the quality of the topology simulation, there is the question 
of simulating dynamic inter- and intra-domain routing. A non-negligible programming 
effort is required if the choice is made not to use a simulator, such as ns II 211 . that has 
these algorithms built in. Finally, the third method is not suitable if routes from a large 
number of sources are to be simulated. Today's route tracing systems employ at most 
a few hundred sources. Caida's skitter 1 17171 infrastructure, for instance, produces an 
extensive graph suitable for simulations, but it based on routes from just thirty sources. 



Note that despite its well known drawbacks, and because of the lack of more ac- 
curate models, the shortest path model is generally used. Examples from recent years 
include Lakhina et al.'s Infocom 2003 paper |22|, Barford et al.'s Sigcomm 2002 pa- 
per |4|, Riley et al.'s MASCOTS 2000 paper l28l . and Guillaume et al.'s Infocom 2005 
paper [15|. The ns network simulator documentation proposes simulating routes by 
shortest paths as an alternative to simulating routing algorithms [ 12 Chs. 26, 29]. 

This paper's principal contribution is a new approach to modeling routes in the 
internet, one that does not share the drawbacks just described. We suggest using an 
actual measured graph of the internet topology, such as the graph generated by skitter. 
From that topology, we suggest choosing sources and destinations as one wishes from 
the nodes of the graph. Between these sources and destinations, we suggest generating 
artificial routes with a model chosen to reflect statistical properties of actual routes. 

Central to this contribution are two specific models for artificial route generation: 
the random deviation model and the node degree model. These models generate routes 
with relatively inexpensive calculations, and the routes that they generate better reflect 
the statistical properties of actual routes than does the shortest path model. 

This paper's other contribution is to update measurements of some familiar statisti- 
cal properties of real routes, notably path length and the hop direction, and to introduce 
and measure a new statistical property: the evolution of node degree along a route. 
These properties serve as the standard for evaluating whether simulated routes resem- 
ble real routes. By introducing this standard, this paper lays the groundwork for going 
beyond the work described here through the eventual introduction of yet better models. 

The remainder of this paper is organized as follows. Sec. |2] describes the data set 
that we have used and the context in which our work lies. Sec. [3]proposes the set of 
statistical properties to describe routes in the internet. Sec. ^proposes the models we 
use to simulate routes based on these properties. Sec. [5] evaluates those models, and 
Sec.|6]concludes the paper. 

2 The framework 

The ideal perspective from which to characterize routes in the internet would be from a 
snapshot of the routing tables of routers throughout the network. Unfortunately, such a 
snapshot is impossible to obtain on the scale of the entire network. In this section, we 
describe the alternative that we opted for, and the hypotheses we made. 

The internet as a graph. Efforts to map the internet graph take place at two levels. 
One is the autonomous system (AS) connectivity graph, which can be constructed from 
BGP announcements (captured for instance by The Oregon Route Views Project |24|). 
The other is the router and IP graph, which can be obtained using traceroute and sim- 
ilar tools from a number of different points in the network. To our knowledge, skitter, 
which conducts traceroutes from on the order of 30 servers to on the order of a million 
destinations, is the most extensive ongoing effort at the IP level. 

Neither level is ideally suited to the task of modeling the behavior of routes at the 
router level. While the AS graph is directly based upon routing information, it is too 
coarse-grained to capture the details of path inflation. For this study, we therefore fo- 
cussed on the IP and router level. 



The main problem with this level is that what one actually sees is the graph of IP in- 
terfaces, while the graph of routers is more relevant. One single node in the router graph 
appears as several separate nodes, one or more for each of its interfaces, in the IP graph. 
Ideally, then, one would construct the router graph using methods to "disambiguate" IP 
addresses, such as the alias resolution techniques described by Pansiot et al. [25], and 
by Govindan et al. 1141 for Mercator. There are also techniques, such as those used by 
Spring et al. [31 29 1, in Rocketfuel, and by Teixeira et al. |35|, that take advantage of 
router and interface naming conventions to infer router-level topology. 

We do not use the router-level graph, however. The disambiguation techniques, as 
applied for example in the iffinder tool from Caida [20 1, do not work by simple in- 
spection of the IP graph; they require active probing, preferably simultaneously with 
graph discovery. This constraint makes extensive disambiguated router-level graphs 
much harder to obtain than IP interface graphs. At best, some core network topolo- 
gies are available in this form thanks to Rocketfuel. But Rocketfuel is untested in stub 
networks. Finally, it is very difficult to judge the extent to which disambiguation is suc- 
cessful, and incomplete or incorrect disambiguation could introduce unknown biases. 

To avoid these difficulties, we have restricted ourselves to the IP graph as obtained 
from skitter. The resulting caveat is that the graph may not be properly representative 
of the router level graph. 

This caveat is however mitigated by the fact that the IP graph nonetheless resembles 
the router graph in one important respect: route lengths are preserved. That is to say 
that a route that has a given length in the router level graph has the same length in the 
corresponding IP graph. Furthermore, as Broido et al. note |6|, "interfaces are individual 
devices, with their own individual processors, memory, buses, and failure modes. It is 
reasonable to view them as nodes with their own connections." 



The data set. This study uses skitter data from July 2 2003. The data was col- 
lected from 23 servers targeting 594,262 destinations. We obtained the corresponding 
IP graph by merging the results of the 7,075,189 traceroutes conducted on that day. This 
graph captures the small-world, clusterized, and scale-free nature of the internet already 
pointed out for instance in numerous publications II19I13I36I37I1I5I . In particular, the 
average distance is approximately 12.54 hops, and the degree distribution is well fitted 
by a power law of exponent 1.97. 

Notice that this graph is necessarily incomplete and biased due in particular to prob- 
ing from a limited number of sources, to route dynamics, to tunneling and to erro- 
neous or absent responses to traceroute probes. Biases of graphs induced by acquisition 
through a small number of traceroute monitors have been studied for instance in by 
Lakhina et al. 1221 . 

However, recent studies by Dall' Asta et al. II II and Guillaume et al. 1 15 1 show that 
one may be quite confident of the accuracy, using this kind of exploration, of distances 
and degrees, which are the main properties that we study here. We therefore consider the 
IP interface graph in this study, and in particular we use the skitter data as it represents 
the current state of the art in its extent and accuracy. 



3 Statistical properties of routes 



This section presents a set of properties for statistical description of internet routes. 
These properties motivate the models of Sec.|4] Several properties have already been 
studied in previous work, and the work here serves to evaluate and update them. 



Route lengths. It is well known that routes are not shortest paths: they are not optimal 
in general. Fig. |l(a)| shows the length distributions of the routes in our data set, and of 
the corresponding shortest paths. It also shows the distribution of the difference (delta) 
between the length of a route and the corresponding shortest path. The mean length of 
15.57 hops for routes in this data set fits closely Paxson's observations |27 26 1 on a data 
set from nine years prior. The shortest paths have a mean length of 12.55 hops (11.4 
hops if the graph is considered to be undirected). 
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Fig. 1: Statistical properties of internet routes. 



The delta distribution confirms Tangmunarunkit et al's observation 1341331 . men- 
tioned at the beginning of this paper, that roughly 80% of routes are not shortest paths. 
In this particular data set, 19.34% of routes are shortest paths. Moreover, since the data 
is incomplete, there are undiscovered links, which implies that 19.34% is an overesti- 
mate. 



Hop direction. When a packet travels from one router to another, it may move closer 
to its destination, but also it may move farther, or it may move to an interface that is at 
the same distance from the destination as one it just left. Likewise, the distance from the 
source may increase, decrease, or stay constant. We will call these behaviors the hop 
direction, considered with respect to either the destination or the source. In principle, 
a hop should always increase the distance from the source and decrease the distance 
to the destination; in such cases, the route is a shortest path. Note that hop directions 
in the router graph can be observed directly in the interface graph, since distances are 
preserved between the two graphs. 

This study observes hop direction by computing the shortest-path hop distance from 
each traceroute source to all other nodes, using breadth-first search. This is feasible due 
to the small number of sources. It would also be natural to look at hop direction with re- 
spect to the destinations but, since they are much more numerous, it is computationally 
expensive. 

We found that 87.3% of hops go forwards, 4.6% go backwards, and 8.1% remain at 
the same distance from the source (we call these stable hops). More precisely, Fig. |l(b)| 
shows the portion of forward, backward, and stable hops at each hop distance for routes 
of 15 hops (the most numerous ones). Note that, as one would expect, the first and 
last few hops are generally forward because there are few alternatives. On the contrary, 
in the core of the network a significant proportion of the hops (more than one third) 
do not go closer to the destination. This type of behavior has already been described 
in the literature as the product of policy-based routing in the core of the internet. As 
Tangmunarunkit et al [34 33 1 note, such behavior may be induced by load balancing, 
commercial considerations, etc. 

Degree evolution along a route. Recent work has shown that many real-world com- 
plex networks tend to have very heterogeneous degrees, well fitted by power laws. This 
is in particular true for the internet, as observed by Faloutsos et al. 1131 and others. 
Moreover, most of the short paths between pairs of nodes in these networks tend to 
pass through the highest degree nodes. Actually, almost all paths (not only short ones) 
tend to pass through these nodes, which make them essential for network connectiv- 
ity eeiitoeiih. 

These observations lead us to ask how the node degree evolves along a route. If 
routes tend to pass through high degree nodes, where do they do so, and what degree 
nodes do they encounter? Furthermore, does this tendency to pass through high degree 
nodes imply that, when a choice exists between next hops, the next hop that leads to the 
highest degree node is generally chosen? 

Fig. |l(c)| shows 3 how node degree evolves for routes of length 15. It reveals that a 
typical route does not pass through the highest degree nodes, though a certain number 
of routes do pass through some very high degree nodes. There is a peak in median out- 
degree observable at distance 1 . The median falls at distance 2, rises again, and then 
stays fairly flat out to distance 13, with a median degree of about 10. This leads us to 
the following interpretation: the hosts have low degree, they are connected at their first 

3 In Fig. | l(c)| dots indicate the median. Vertical lines run from the min to Ql and from Q3 to 
the max. Tick marks indicate the 5 th , 10 th , 90 th and 95 th percentiles. 



hop router to relatively high degree nodes which play the role of access points, and then 
packets are routed in a core network where the degree (typically 10) does not depend 
much on the distance from the source or from the destination. 

One may wonder if there is a simple local rule that can be observed for the degree 
evolution. In particular, if there is a choice of next hop interface along a route, is there 
a correlation between the degree rank of an interface and its probability of being cho- 
sen? For instance, are higher degree interfaces chosen preferentially over lower degree 
ones? Note that such a rule could be perfectly compatible with the observed flat degree 
evolution. 

Fig. |l(d)| plots the probability that a packet travels to an interface's i-th ranked 
neighbor, where the neighbors are ranked from highest out-degree to lowest. An inter- 
face's neighbors are its possible next hops in the directed graph. In order to preserve the 
greatest detail in this middle range, the figure does not show curves for degrees 2 or 3, 
or above 10, but the curves shown are typical. 

For instance, when an interface has five possible next hops, the probability that the 
next hop along a route will be the highest ranked neighbor is 0.22. The probability 
that the next hop will be the second ranked neighbor is 0.21. Probabilities continue to 
decrease, and the fifth ranked neighbor is chosen with a probability of 0.18. One can 
see a clear bias towards highest degree nodes, though this bias is rather small. 

4 Route models 

The previous section provides a set of simple statistical tools to capture some properties 
of routes in the internet. We now propose three simple models (only two of which we 
eventually retain) designed to capture these features. Each model is based upon one 
statistical property studied in the previous section. Our approach is to model a property 
in a very simple way and then use other statistics to validate or invalidate the model. 

Whereas our study of route properties was in the context of the directed graphs pro- 
duced by traceroute, the models in this section are proposed for undirected graphs. The 
graphs available for simulation purposes, notably those produced by topology genera- 
tors representing the router-level topology, are typically undirected graphs. Therefore, 
our models must be suitable for use in this context. 

Path length model. The path length model is the simplest and the most obvious one 
conceptually, but it proves to be unusable in practice. The model aims at producing 
routes of the same lengths as real ones. As discussed in Sec. |5J a real route length 
typically exceeds that of the shortest known path by some small integer value 5 ^ 0. 

In order to construct a route from a source s to a destination d, the path length model 
first computes the length £ of a shortest path from s to d. Then it samples a deviation 

5 from a distribution such as the one shown in Fig. |l(a)| and a route is generated by 
choosing a path at random from s to d among the ones which are loop-free and have 
length I + S, This ensures that the difference between shortest path lengths and actual 
route lengths will be captured by the model. 

To choose such a path at random implies however that one must construct all of 
the loop-free paths of length 1 + 5 from s to d. In practice, the computation required 



to generate this number of paths may be prohibitive, since even in simple cases it is 
exponential in I + 6. For example, in trying to generate all paths of length 21 between a 
pair of nodes in the skitter graph, we enumerated 1,206,525 possible paths. Therefore, 
despite its simplicity, we will not consider this model further. 

Random deviation model. The random deviation model is based upon the idea that a 
route usually follows a shortest path, but might occasionally deviate from it. We mod- 
eled this using one single parameter, p, the probability at any point of deviating from 
the current shortest path to the destination, if such a deviation is possible. We tuned the 
value of p to generate routes of realistic length. For the undirected version of the skitter 
graph, we found p = 0.2 to work well. 

A random deviation route from source s to destination d is therefore based upon a 
shortest path u from s to d. At each hop, with probability 1 — p, the route continues 
along u. But with probability p it will, if possible, deviate off u to another path. A 
deviation from current node a; to a neighboring node y is deemed possible only if there 
is a shortest path w from y to d that does not pass through x. Should there be a deviation, 
the route continues along w to d (unless another deviation should occur). The model is 
precisely described by Algorithm^ 

Note that large numbers of routes to a given destination d can be efficiently gen- 
erated with the random deviation model once a shortest path tree rooted at d has been 
computed. 



Algorithm 1: rand_dev_route (G,s,d,p) 

Input : A network G, a source s, a destination d, a deviation probability p. 
Output : An artificial route v from s to d in G, following the random deviation model. 
Function: sp(x.y) returns the set of all the shortest paths from x to y in G. 
1 begin 
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u 




andom element of sp ( s , d ) ; 
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— empty list; 
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copy the first element of u to the end of v\ 
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remove it from u; 
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while the last element ofv is not d do 
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if rand[0,l] ^ p then 
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C <— set of all the shortest paths from any neighbor of v to 
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Remove from C the paths containing the last element of v; 
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if C ^ then 
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|^ u random element of C; 
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copy the first element of u to the end of v; 


13 




remove it from u\ 


14 


return v; 


15 end 







Node degree model. Several previous authors |38 21 3| have tried to use the hetero- 
geneity of node degrees to compute short paths in complex networks. The basic idea is 
that a path which goes preferentially towards high degree nodes tends to see most nodes 
very rapidly (a node is considered to be seen when the path passes through one of its 
neighbors). 

The node degree model is based upon a similar approach, as follows. Two paths 
are computed, one starting from the source and the other from the destination. The 



next node on the path is always the highest degree neighbor of the current node. The 
computation terminates when we reach a situation where a node is the highest degree 
neighbor of its own highest degree neighbor. One can show that this is the only kind of 
loop can occur. Then, one of two cases applies: either the two paths have met at a node, 
or they have not. In the first case, the route produced by the model is the discovered path 
(both paths are truncated at the meet up node, and are merged). In the second case, we 
compute a shortest path between the two loops, and then obtain the route by merging 
the two paths and this shortest path, removing any loops. 

This method has already been proposed 1 3 1 as an efficient way to compute short 
paths in complex networks in practice: the obtained paths are very close to shortest 
ones. Moreover, the computation of the tree-like structure where each node points to its 
highest degree neighbor is very simple and only has to be processed once. Likewise, the 
shortest paths between a small number of loops are computed only once. The overall 
model is described in Algorithmic 



Algorithm 2: node_deg_route (G,s,d) 

Input : A network G, a source s, a destination d. 

Output : An artificial route v from s to d in G, following the node degree route model. 
Function: reverse(p): returns the path obtained by reading p from the end to the beginning. 

climb_degrees(G,v): returns the path in G obtained from v by going to the highest degree neighbor at 
each hop, until it loops. 

1 begin 

2 p s i — climb_dcgrces (G,s); 

3 pa < — climb.dcgrccs (G,d); 

4 if p s and Pd meet up then 

5 let u be the first node they have in common; 

6 remove from p s all the nodes after u; 

7 remove from pd all the nodes after u; 

8 p <— (p s ,reverse(p,j)); 

9 return p; 

10 q < — random shortest path from the last node of p s to the one of pa ; 

11 p < — (p s ,q,reversc(pd))\ 

12 remove loops from p; 

13 return p; 

14 end 




d 



.. s 

Fig. 2: The node degree model: example. 



Fig. |3 is an example. There are three tree-like structures (the shaded areas). The 
source s belongs to the leftmost one, which is rooted at r s , and the destination d to 
the rightmost one, with root at r^. Each directed link goes from one node to its highest 
degree neighbor (the dotted lines are links which do not satisfy this). When one wants 
to build a route from s to d according to the node degree model, one first finds the path 
from s to r s , and the one from d to r^. One then has to compute a shortest path from 
r s to rd, which has length 5 in this example. The final route is obtained by merging 
these paths, and then removing the loops (which leads to the removal of a link, in our 
example). It has length 7 (while the shortest path has length 6). 



5 Evaluation 

This section compares the performance of the random deviation and node degree mod- 
els to that of the shortest path model. We use undirected version of the skitter graph 
described in Sec.|2] considered as an undirected graph. For each model, we chose at 
least 60,000 (source, destination) pairs at random from amongst the nodes of the graph 
and generated an artificial route from the source to the destination. We compute the 
same statistics on these routes as we had computed for actual routes in Sec. [3] 

Fig.|3]shows the statistics for each model. We judge the quality of a model by how 
well its statistics mirror those for actual routes, shown in Fig. [2 

Comparing the route length distributions, we find that both models generate distri- 
butions that are symmetric, average somewhat higher than the shortest path distribution, 
and have tails similar to the actual route length distribution shown in Fig. |l(a)| Mean 
route length is 15.15 for the random deviation model and it is 14.96 for the node degree 
model, whereas the mean shortest path is 12.93. (Note that, on the undirected skitter 
graph, shortest paths between random sources and destinations are longer on average 
than those between skitter sources and destinations, for which we had computed an 
average route length of 11.21.) 

Lengths of paths generated with the node degree model tail off somewhat quicker 
than in reality (approaching zero closer to length 20 than length 25), but the degree of 
fidelity is nonetheless remarkable given that the length distributions are not explicitly 
part of the model. The random deviation model generates more routes that are shortest 
paths than in reality (roughly 30% compared to roughly 20%), whereas the node degree 
model generates somewhat fewer (roughly 26%). As is already known, the shortest path 
model does not capture the length properties. 

Looking at the hop directions for the most frequent route length, we found that 
the curves for the random deviation model better match the shapes of the curves for 
real routes shown in Fig. |l(b)| Hops are mostly forward near the source, but dip to 
around 80% roughly ten hops out (whereas in reality the portion of forward hops dips 
to around 80% at eleven or twelve hops out). This is in marked contrast to hop directions 
produced by the node degree model because forward hops dip much sooner and a bit 
less steadily. But overall portions of forward, stable, and backward hops closely match 
reality for both models: 89% forward, 7% stable, and 4% backward for the random 
deviation model, and 90% forward, 6% stable, and 4% backward for the node degree 
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Fig. 3: Experiments using the random deviation model (left), the node degree model (center), and 
the shortest path model on the undirected skitter graph using sources and destinations chosen at 
random from amongst all the nodes in the graph. 

model, compared to 87% forward, 8% stable, and 5% backward for true routes. The 
shortest path model fails to capture these proportions since all of its links are forward. 

The node degree model shines compared to the random deviation model in capturing 
the evolution of the out-degree close to a route's source. Routes generated with this 
model show the peak in the out-degree before settling down to a median around 20 that 
we noticed in Fig. |l(c)| though the peak is reached at distance 2 rather than at the first 
hop router. The random deviation model and the shortest path model also have a median 
around 20, but they arrive there through a smooth increase, with no clear peak. 

Based upon this comparison to real routes, we can state that the random deviation 
and node degree models do a reasonable job of emulation, though each model captures 
some aspects better than others, and their strengths are different. Both models clearly 
out-perform the shortest path model. 



6 Conclusion and future work 



The main contribution of this paper has been to propose a new alternative for the simula- 
tion of routes in the internet: the use of simple models that capture non-trivial statistical 
properties of routes. The models proposed here have been found to reproduce a number 
of aspects of true internet routes, though neither fully captures all of the characteristics. 
Our goal was to introduce simple models that could serve as alternatives to the clearly 
unrealistic shortest path model. No model can be fully faithful to reality, and the key 
is to understand in what ways it is a true representation, and in what ways it diverges. 
Future work along these lines might include the development of models that explicitly 
incorporate some additional characteristics, such as the clustering coefficient. Other 
work might involve studying whether certain variants on the models, such as a hybrid 
of the random deviation and node degree approaches, would be more like real routes. 
Any such work must keep in mind the desirability of keeping the models conceptually 
simple, easy to implement, and computationally tractable. 

We have shown how routes can be simulated on a measured graph at the interface 
level. We have chosen the undirected variant of the skitter graph, as undirected graphs 
are more readily available for simulation purposes. We have also introduced the simpli- 
fying assumption that any node can be either source or destination. Using graphs that 
contain direction information and that label end-hosts separately from routers could po- 
tentially improve the quality of the models. Also, these same models can potentially be 
used on synthetic graphs that are meant to represent the internet topology. 

Another area into which this work could be extended would be to capture something 
of the dynamics of internet routes. There are effectively random choices to be made in 
both the random deviation model (clearly) and the node degree model (when it comes 
to choosing among two or more neighbors of highest degree, or choosing a shortest 
path between two trees). Therefore these models may produce routes that vary between 
a given source and a given destination. But we have not touched on the timing of that 
variation, the topological relatedness or lack thereof of consecutive routes, or the man- 
ner in which path lengths change over time. Much remains to be done in this promising 
direction. 
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